Location-Independent Transformations: a General Strategy for Implementing Neural Networks

Most Artificial Neural Networks (ANNs) have a fixed topology during learning, and typically suffer from a number of shortcomings as a result. Variations of ANNs that use dynamic topologies have shown ability to overcome many of these problems. This paper introduces Location-Independent Transformations (LITs) as a general strategy for implementing neural networks that use static and dynamic topologies. A LIT creates a set of location-independent nodes, where each node computes its part of the network output independent of other nodes, using local information. This type of transformation allows efficient support for adding and deleting nodes dynamically during learning. Two simple networks, the single-layer competitive learning network, and the counterpropagation network, which combines elements of supervised learning with competitive learning, are used in this paper to illustrate the LIT strategy. These two networks are localist in the sense that ultimately one node is responsible for each output. LITs for other models are presented in other papers.